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[57] ABSTRACT 

In a method of estimating motion per picture portion in 
an image of a picture signal, wherein, starting from a 
first (x0=xl, y0=y2) and a second (x0=x3, y0=y4) 
starting vector a motion vector is determined, a first and 
a second candidate motion vector, respectively, are 
determined on the basis of the first and second starting 
vector in accordance with a predetermined criterion, a 
motion vector is selected from both candidate motion 
vectors, and as components of the starting vectors, 
respective corresponding components of candidate mo- 
tion vectors (xl, yl), (x2, y2), (x3, y3) and (x4, y4) al* 
ready previously determined are taken, which candi- 
date motion vectors correspond respectively to the 
starting vectors. 

20 Claims, 2 Drawing Sheets 
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use twice the number of circuit elements for the realisa- 

METHOD OF ESTIMATING MOTIION IN A tion of the dual motion estimation, compared to a single 
PICTURE SIGNAL motion estimation, because only half the number of 

picture elements are left for each group because of this 
BACKGROUND OF THE INVENTION 5 sub-sampling operation, more specifically a different 

The invention relates to a method of estimating mo- each phase. Therefore, in this embodiment each 

tion of a picture portion in an image of a picture signal, P^ure portion comprises at least two picture elements, 
comprising determining a number of candidate motion A further embodiment of the method according to 
vectors starting from a corresponding number of start- the invention is characterized in that a first component 
ing vectors, and selecting a motion vector out of the 10 of a first starting vector is taken to be equal to a first 
determined candidate motion vectors. component of a first candidate motion vector already 

Such a method is described in U.S. Pat. No. previously determined for a first picture portion, a sec- 
4,853,775. Therein four starting vectors give rise to four ond component of said first starting vector is taken to be 
candidate motion vectors, and only that motion vector equal to a second component of a second candidate 
is selected, out of these four candidate motion vectors, 15 motion vector already previously determined for a sec- 
which gives a convergence of the used gradient algo- ond picture portion, while the first and second picture 
rithm with the smallest frame difference. The four start-. portions do not coincide, and in that a first component 
ing vectors correspond to motion vectors already esti- of a second starting vector is taken to be equal to a first 
mated for four neighboring pixels in the causal neigh- component of a third candidate motion vector already 
borhood of the present pixel. 20 determined for a third picture portion, a second compo- 

, SUMMARY OF THE INVENTION nent of said second starting V £^S 

to a second component of a fourth candidate motion 

The invention has inter alia for its object to provide vector already determined for a fourth picture portion, 
an improved motion estimation. while the third and fourth picture portions do not coin- 

A method of the type defined in the opening para- 25 cJde the startm g vectors are syntheses of compo- 
graph is therefore characterized by taking as compo- nenu G f tw0 different candidate motion vectors already 
nents of the starting vectors, components of candidate determined, and the risk that on passing beyond a con- 
vectors already previously determined, which candi- m ^ picture| both components of the starting 
date motion vectors correspond respectively to the VfiCtors bd tQ a different ohiect than the ob jec t the 
starting vectors, and which candidate motion vector 3U present picture part be longs to, becomes smaller so that 
components correspond respectively to the startmg g qufckef ^Mergence to a correct (candidate) motion 
vector components. veclor be achieved The f irst picture portion ma y f 

Unless explicitly stated difTerently m this description * be located opposite the third picture por- 

a picture portion may consist of a ^Pf^^ £ £ ure ^ for F which ^ 

or of a group (far ^example a block) of picwe eXem^ 35 P ^g^^ icture 

Because according to the invention, instead of the . , 6 * \- , 

, , °. . . . . ' .. .,„„.„, „ f aon may be opposite the fourth picture portion in rela- 

components of an ultimately seated monon vector of * ic ^ reportion for wh ? c h ^ motion is being 

a neighbonng p IC ture portion, respect, yely correspond- P £ independence between the 

mg components of at least one candidate motion vector * , v " . j 

determined for that neighboring picture portion are 40 two estimation resulfc is obtained, 

taken as components of each starting vector, which ^ embodiment thereof is characterized in that the 

candidate motion vectors correspond respectively to third and fourth picture portions coincide with the 

the starting vectors, the advantage is obtained that a ^nd and first picture portions, resp^tively In the 

convergence direction associated with each starting e*ampl e mentioned in the foregoing m which the first 

vector component, for example from top to bottom or 45 Picture portion was opposite the third portion, and the 

from left to right in the picture, is fully calculated, even second picture portion opposite the fourth portion, two 

if a candidate motion vector used for composing a start- Picture portions are to the right of or lower in the pic- 

ing vector did not lead to the best results for its own ture than the picture portion for which the motion was 

picture portion. However, it has been found that the being estimated. Because motion vectors must already 

motion vectors thus obtained are better than the motion 50 be available for these two picture portions, a motion 

vectors based on starting vectors which were the ulti- estimation must also be carried out from the bottom to 

mately selected, best motion vector of a neighboring the top in the picture or motion vectors determined for 

picture portion. A candidate motion vector which cor- a previous picture should be used. For the realization of 

responds to a starting vector and is already previously this motion estimation, either the picture elements of the 

determined for, for example, a left-hand adjacent pic- 55 picture or the motion vectors must be stored in a mem- 

ture portion, is to be understood to be that candidate ory, so that an additional memory is required. This 

motion vector that is based on a starting vector which, additional memory is not required in the present em- 

in its turn, is a candidate motion vector of a left-hand bodiment, since, if there are only two picture portions 

adjacent picture portion of said left-hand adjacent pic- the estimated motion vector of which is used as a start- 

turc portion. 60 ing vector for a new estimation, these picture, portions 

An embodiment of the motion estimating method can be placed such in relation to the picture portion for 

according to the invention is characterized in that pic- which the motion is being estimated, that only a motion 

ture elements of each picture portion are divided into a estimation from top to bottom in the picture is neces- 

first and a second respective group by sub-sampling the sary. 

picture portion in a first and a second phase, a first 65 A still further elaboration of this embodiment, is char- 
candidate motion vector being determined for the first acterized in that the first and second picture portions 
group and a second candidate motion vector being have a smaller y-coordinate than the picture portion for 
determined for the second group. It is not necessary to which the motion is being estimated, the first picture 
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having a smaller x-coordinate and the second picture in a corresponding decoding section a motion-compen- 

portion having a larger x-coordinate than the picture sated interpolation by means of transmitted vector in- 

portion for which the motion is being estimated. The formation is performed. 

x-coordinate increases from left to right, and the y-coor- A further embodiment of the method according to 
Hinatp increases from top to bottom. Thus, compared 5 the invention is characterized in that previous to the 
with a possible embodiment in which the first picture motion estimation, the picture signal is temporally low- 
portion is directly to the left of the picture portion for pass filtered by means of a temporal filter with coeffici- 
which the motion is being estimated, the advantage is ents (i, J, J), wherein from field to field lines from both 
obtained that a larger time interval now becomes avail- neighbouring fields located alternately above ot below 
able between the moment at which the first candidate 10 a line in a present field are multiplied by the coefficient 
motion vector can be defined and the moment at which i. Thus, the advantage is obtained that high vertical 
this estimation result is needed again to obtain the start- frequencies in a picture which might cause large differ- 
ing vectors belonging to the present picture portion for ences between subsequent fields in the picture, which 
which the motion is being estimated. are not caused by movement, are filtered out, so that a 

A different embodiment according to the invention is 15 better motion estimation is obtained. In practice the 
characterized in that the picture portions of a picture filter used appears to provide good results, 
for each row of picture portions are shiftedly stacked Another embodiment of the method according to the 
blocks of picture elements. This appeared to be a very invention is characterized in that said candidate motion 
suitable choice especially in connection with the still vector determining step includes the steps of: 
further elaboration described in the foregoing. 20 taking the median of said picture signal, a field- 
According to, for example, the "One step at a Time delayed picture signal and a field- and line-delayed 
Search" (OTS) criterion, disclosed in IEEE Transac- picture signal to obtain a filtered signal; and 
tionson Communications, No. 8, 1985, pages 888-896, estimating said candidate motion vectors based on 
the estimated motion vector may be equal to the starting said picture signal and said filtered signal. Due to this 
vector. An embodiment of the method according to the 25 median prefilter, a kind of quasi progressive scan signal 
invention is characterized in that the candidate motion is applied to the proper estimator, so that estimation 
vectors are determined in accordance with a criterion errors caused by interlace are avoided, 
that provides that only a candidate motion vector According to another aspect of the invention, there is 
which differs from the starting vector can be deter- provided a method of estimating motion per picture 
mined. Because of this, the algorithm is prevented from 30 portion in an image of a picture signal wherein, starting 
remaining in a local minimum. from a starting vector, a motion vector is determined 

A still further embodiment of the method according according to a previously defined criterion, character- 
to the invention, is characterized in that components of ized in that a first component of the starting vector is 
the estimated motion vectors are spatially low-pass taken to be equal to a first component of a first already 
filtered. This provides a vector field without abrupt 35 previously, estimated motion vector, belonging to a first 
transients, which inspite of an inaccuracy caused by the picture portion, and a second component of the starting 
low-pass filtering, appeared to lead to a finer result, vector is taken to be equal to a second component of a 
because block-shaped errors in a picture interpolated by second, already previously estimated motion vector, 
means of the vectors obtained occur less frequently. belonging to a second picture portion, said first and 
Because of the averageing effect of the low-pass filtra- 40 second picture portions not coinciding. This aspect of 
tion, a sub-pixel accuracy may even be .obtained. Sub- the present invention is based on the recognition that no 
pixel accuracy must be understood to mean an accuracy - optimum estimation result will be obtained, when dur- 
which is finer than the distance between subsequent ing the estimation of a motion vector for a picture por- 
picture elements. tion on one side of a contour a starting vector estimated 

An elaboration thereof is characterized in that said 45 for a picture portion on the other side of the contour 

low-pass filtering step includes the steps of and which therefore probably belongs to a different 

recursively filtering said selected vector components object, is used as the starting point. The measure ac- 

to obtain output components; cording to this aspect of the invention, which provides 

comparing said output components to said selected that the starting vector is a synthesis of components of 

vector components to obtain a control signal indicating 50 two different already previously estimated vectors, 

whether said selected vector components differ more reduces the risk that during passing beyond a contour in 

than a given threshold from said output vector compo- the picture both components of the starting vector be- 

nents; and long to the other object, so that a faster convergence to 

disabling said recursively filtering step if said control a correct motion vector can be obtained, 

signal indicates that said selected vector components 55 An embodiment of this aspect of the invention is 

differ more than said given threshold from said output characterized in that for each picture portion not only 

vector components. Due to the disabling of the postfil- said motion estimation, but also a second motion estima- 

tering operation when the difference between a newly tion is effected, in which for the second motion estima- 

estimated vector and a filtered output vector becomes tion components of a second starting vector are as : 

larger than a given threshold, it is avoided that vectors 60 sumed to be equal to corresponding components of 

belonging to different objects in the scene and moving third and fourth, already previously estimated motion 

in different directions are averaged. vectors, respectively, which belong to third and fourth 

A more specific embodiment thereof is characterized non-coinciding picture portions, respectively, and a 

in that the low-pass-filtered vector components are second motion vector is determined starting from the 

submitted to a spatial sub-sampling operation which 65 second starting vector according to said criterion, 

provides a saving in data, so that the method according whereafter from said first and second determined mo- 

to the invention can, for example, be used in an en cod- tion vectors, a best motion vector is selected. This pro- 

tng section of a transmission or storage system, whereby vides the advantage that the estimation result depends 
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to a less extent on structures (contours) present in the vectors used are finally selected motion vectors. A 
picture because the best motion vector, i.e. the motion candidate motion vector which corresponds to a start- 
vector which results in the smallest estimation error, is : ing vector and is already previously determined for, for 
selected from two separately defined candidate motion example, a picture portion adjacent to the left, must be 
vectors, which were obtained by means of different 5 understood to mean that candidate motion vector that is 
starting vectors. In this situation, in relation to the pic- based on a starting vector which, in its turn, is a candi- 
ture portion for which the motion is being estimated, date motion vector of a picture portion adjacent to the 
the first picture portion may be opposite to the third left thereof. Consequently a convergence direction be- 
picture portion and the second picture portion may be longing to each starting vector, for example, from top 
opposite the fourth picture portion, whereby an optimal 10 to bottom or from left to right in the picture, is fully 
independence betwen both estimation results is ob- calculated, even if a candidate motion vector thus ob- 
tained. It should be noted that it is known per se from L. tained does not provide the best results. However, it 
Looijenga's graduation report, "Methodes voor bewe- was found that motion vectors thus obtained are better 
gingscompensatie in een sequentie van beelden", TU than motion vectors based on starting vectors which 
Delft, 1986, pp. 27-28, to select a starting vector from 15 were the ultimately selected, best motion vectors of a 
two candidate starting vectors by comparing which of previous picture portion. It is thus necessary for each 
these, as a motion vector, results in the smallest estima- picture portion to have both starting vectors lead to a 
tion error, to determine subsequently a motion vector, candidate motion vector, in order to be able to start 
starting from the selected starting vector according to from candidate motion vectors, 
the previously defined criterion. 20 A picture portion may be either a single picture ele- 

These and other (more detailed) aspects of the inven- ment or a group (block) of picture elements. The previ- 
tion wfll now be described and elucidated in greater ously defined criterion may be, for example, the **One 
detail with reference to the following non-limitative step at a Time Search" (OTS) criterion mentioned in 
. examples and drawings. IEEE Transactions on Communications, No. 8, 1985, 
. ■ T „ 25 pages 888-896, or the "Gradient Adaptive Search" 

BRIEF DESCRIPTION OF THE DRAWINGS Sctoicm mentioned in the ** Intern cif. on Digital 

FIG. 1 shows a first possible configuration of picture Processing of Signals in Communications, Loughbor- 
portions with associated vector components; ough, 20-23 September 1988, publication No. 82, Insti- 

FIG. 2 shows a second possible configuration of pic- tution of Electronic and Radio Engineers, page 
ture portions with associated vector components; 30 109-1 15. The OTS criterion implies that a shift with the 

FIG. 3 shows a configuration sampled in a first and a smallest possible associated • estimation error is first 
second phase, of shiftedly stacked blocks of picture looked for in the horizontal or x-direction, starting from 
elements; the starting vector, and subsequently, starting from this 

FIG. 4 shows the mode of operation of a temporal shift, a shift with the smallest possible associated estima- 
prefilter with reference to a sketch of subsequent rasters 35 tion error in the vertical or y-direction is looked for. 
assembled from lines; The previously defined criterion may be a criterion in 

FIG. 5 shows a possible structure of a spatial prefilter which the starting vector itself is a possible estimation 
or postfilter; result, or a criterion in which the starting vector itself 

FIG. 6 shows an embodiment of a motion estimator may not be an estimation result. If the starting vector 
according to the invention; 40 itself cannot be an estimation result, less circuit elements 

FIG. 7 shows part of another embodiment of a mo- are required for a criterion which compares estimation 
tion estimator according to the invention, in which a errors of a plurality of candidate motion vectors, be- 
median prefilter is placed before a candidate motion cause the estimation error occurring when using the 
vector determining circuit; and starting vector now need not to be defined and com- 

FIG. 8 shows a spatial postfilter for the estimated 45 pared. Furthermore, a criterion in which the starting 
motion vectors. vector itself is excluded, prevents the algorithm from 

«™ . „ ™ *™,^ T „ _ ^ TOT . getting stuck in a local minimum. By using such a crite- 

DE ^^^,P^^2^U^^ D Hon, the motion estimation will result to a lesser extent 
EMBODIMENTS OF THE INVENTION m a estimation result for subsequent picture 

FIG. 1 shows a first possible configuration of picture 50 portions, which however is no problem because a spa- 
portions with associated vector components. In the tial low-pass filtering of the vector components may 
lower right-hand corner of FIG. 1, it is shown how the remove oscillations caused by the criterion around a 
x- and y -coordinates are defined in the description and correct vector value. 

in the claims. The x -coordinate increases therein from When larger block sizes are used, a criterion which 
left to right, and the y-coordinate from top to bottom. 55 selects the vector having the smallest error on the axis 
In the middle of FIG. 1, a picture portion is indicated having the largest fault gradient, was found to furnish 
for which a motion vector is to be selected from two good results. A criterion according to which four vec- 
candidate motion vectors, which must be defined start- tors (a±k, b) and (a, b±k), with ±k picture element 
ing from two starting vectors (xO, yO) according to variations in either the horizontal or the vertical direc- 
some previously defined criterion. The starting vectors 60 tion relative to a starting vector (a, b) are compared in 
are composed of the components of candidate motion order to determine which vector provides the best 
vectors' already determined for other picture portions; match, also provides good results and, moreover, is 
for each starting vector at least one corresponding can- easier to realise. In general it holds that the oonver- 
didate motion vector is used. It is to be noted that candi- gence rate may increase with increasing values for k but 
date motion vectors determined by estimations for 65 may sooner result in instabilities. In practice it appears 
neighboring picture portions are used to compose the that an optimum value for k equalled 1 when a picture 
starting vectors for the estimation of the current picture portion size of 1 X2 picture elements is used, and that 
portion 4>; it is irrelevant whether the candidate motion the optimum value for k equalled 2 when a picture 
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portion size of 4x 8 is used. It is alternatively possible to 
have the value k depend on the value of the estimation 
error, such that for a relatively large error a higher 
value for k is being taken, so that a different and, as is to 
be hoped, better vector may be found faster, while for a 
relatively small error a small value for k is sufficient, as 
the vector is apparently already fairly correct Further- 
more, it is possible to compare eight vectors obtained by 
means of values for k equal to 1 and 3, in one operation. 

In FIG. 1, first I and second II picture portions, re- 
spectively, are shown to the left of and above the cen- 
tral picture portion 4>, for which a first candidate mo- 
tion vector (xl, yl) originating from the left and a sec- 
ond candidate motion vector (x2, y2) originating from 
above, respectively, have already been determined. In a 
very straightforward implementation, the candidate 
motion vector (xl, yl) could be taken as a first starting 
vector of the central picture portion <f>, and the candi- 
date motion vector (xi, y2) could be taken as a second 
starting vector of the central picture portion <(>• How- 
ever, it is preferred that during determining the first 
candidate motion vector of the central picture portion 
<J>, a first starting vector is taken, the components of 
which originate from two already previously deter- 
mined candidate motion vectors, for example (xO^xl, 
y0=y2). The second starting vector may be the vector 
(x0=x3, y0=y4), wherein the components x3 and y4 
originate from a third candidate motion vector (x3, y3) 
already determined, and a fourth candidate motion vec- 
tor (x4, y4) already determined, respectively, which 
belong to respective third m and fourth IV picture 
portions. Compared to the central picture portion <{> for 
which the motion is being estimated, the third III and 
fourth IV picture portions are located opposite the first 

- picture portion I and the second picture portion II, 
respectively, so that an optimum independence between 

. the two candidate motion vectors to be determined is 
obtained As each starting vector is a synthesis of com- 
ponents of two candidate motion vectors already deter- 
mined and belonging to two different picture portions, 
the risk is smaller that during passing .through a contour 
in the picture, both components of the starting vector 
belong to a different object than to which the present 
picture portion belongs, so that a faster convergence to 
a correct motion vector can be obtained. Each motion 
vector has its own convergence direction, in the exam- 
ple given the first starting vector component converges 
in the x -direction and the second starting vector compo- 
nent converges in the y -direction. Starting from each of 
these starting vectors, a candidate motion vectors is 
determined according to some previously defined crite- 
rion which starts from the corresponding starting vec- 
tor. 

It is to be noted that the recognition that a starting 
vector of a motion estimation method preferably is 
composed of components derived from two motion 
vectors already estimated for two non-coinciding adja- 
cent picture portions, is generally applicable and not 
limited to use in combination with the recognition that 
a starting vector preferably is composed of components 
derived from at least one candidate motion vector in- 
stead of from at least one finally selected motion vector. 

A second possible configuration of continuous pic- 
ture portions with accompanying vector components is 
shown in FIG. 2. This configuration, in which the first 
picture portion I is located obliquely above the picture 
portion <J> for which the motion must be estimated, has 
the advantage, that compared with the configuration 



shown in FIG. 1 in which the first picture portion I was 
located on the same line or lines as the picture portion <J> 
for which the motion must be estimated, there is a larger 
time interval between the instant at which the candidate 
5 motion vector (xl, yl) is determined and the instant at 
which this candidate motion vector is required for the 
composition of the starting vector (xO, yO). The starting 
vector may be compounded in the same manner as 
described in relation to FIG. 1. The starting vector may 
10 also be the vector (xO=xl, y0=y2) for determining the 
first candidate motion vector and (x0=x2, yO=yl) for 
determining the second candidate motion vector. This 
has the advantage that it is only necessary to proceed 
from the left upper corner of the picture, and not also 
15 from the right lower corner or using previous field 
motion vectors, to have estimation results of the candi- 
date motion vectors (x3, y3) and (x4, y4) available to 
obtain the second starting vector, while there are still 
two sufficiently independently defined starting vectors. 
20 Another solution for the non-causality problem that 
arises if the candidate motion vectors of picture por- 
tions III and IV shown in FIGS. 1 or 2 are used, is to use 
the candidate motion vectors determined for these pic- 
ture portions during the previous field period. 
25 If desired, it is also possible to compare the candidate 
motion vectors determined by starting from 4 starting 
vectors which are the corresponding candidate motion 
vectors of all picture portions I-IV, while the candidate 
motion vectors of picture portions III and IV are again 
30 those determined during the previous field period. In 
another modification, each starting vector may be com- 
posed of the components of two candidate motion vec- 
tors already determined for the adjacent picture por- 
tions 1-IV. Due to these modifications, a faster conver- 
35 gence of the algorithm can be obtained on passing be- 
yond a contour in the picture, because estimation results 
from both sides of the contour are taken into account. 

According to yet another modification, it is possible 
to select between an estimation based on previous field 
40 vectors and an estimation based on current field vec- 
tors. In this modification, the estimation based on previ- 
ous field vectors may use the configuration of FIG. 1, in 
which the candidate motion vectors of all picture por- 
tions I-IV are used. The estimation based on current 
45 field vectors may use the configuration of FIG. 2, in 
which only the candidate motion vectors of picture 
portions I and II are used. The selection between the 
estimation based on previous field vectors and the esti- 
mation . based on current field vectors is preferably 
50 based on stored estimation errors which resulted when 
the candidate motion vectors (xl, yl) were applied to 
their own picture portions I of FIG. 1 and of FIG. 2, 
respectively. As the selection is an a priori selection, i.e. 
based on data which are known before either of the 
55 estimations is actually performed, it is not necessary to 
perform the non-selected estimation. It is of course also 
possible to base the selection on a comparison of the 
results obtained by the estimation based on previous 
field vectors and the estimation based on current field 
60 vectors, i.e. an a posteriori selection, but then more 
processing equipment is required. The estimation based 
on previous field vectors can be simplified if among the 
previous field vectors, only the candidate motion vector 
determined for the picture portion IV of FIG. 1 is taken 
65 into account, so that altogether picture portions I and II 
of FIG. 2 and picture portion IV of FIG. 1 are used. 

A configuration of shiftedly stacked blocks of 
two-by-four picture elements, sampled in a first and a 
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second phase is shown in FIG. 3. The samples resulting age system, having only a limited capacity for the trans- 
from the sampling operation in the first phase are indi- mission of motion information for effecting a motion- 
cated in FIG. 3 by a *T*, and the samples resulting from compensated interpolation by means of the transmitted 
the sampling operation in the second phase are indi- vector information, in a corresponding decoding sec- 
cated in FIG. 3 by a "2". As only half the original num- 5 tion. With an interpolation of the picture signal by 
ber of picture elements remains for each phase in this means of vector components, recovered through inter- 
sampling operation, it becomes possible to carry out the polation from the transmitted vector information, per 
dual motion estimation with different starting vectors small picture portions, block errors caused by block 
according to the invention, without it being necessary boundaries of large blocks, not coinciding with object 
to use twice the number of circuit elements. Herein, use 10 boundaries are decreased, while, because of the sub- 
is made of starting vectors which are composed of vec- sampling operation of the vector components, no more 
tor components of candidate motion vectors already vector information is being transmitted than would be 
determined, belonging to blocks which are indicated by transmitted if the motion would be estimated for larger 
I and II in the method illustrated in FIG. 2 for the esti- blocks. 

mation of the motion of a block denoted by <f>. For 15 A possible structure of a suitable spatial low-pass 
example, the starting vector (x0= xl, yO = y2) is used for filter for the filtering operation, previous to the motion 
the first phase, and the starting vector (x0=x2, y0=yl) estimation, of the picture signal and/or for filtering of 
is used for the second phase. It is also possible to use the the estimated vector components, is shown in FIG. 5. 
starting vector (x0=x3, y0=y4) for the second phase, An input 1 of the filter is connected to a first input 3 of 
in which use is made of vector components of motion 20 an adder 5, of which an output 7 is connected to a sec- 
vectors already estimated, belonging to blocks that are ond input 9 of the adder 5 via a memory P with a delay 
indicated by III and IV. Obviously, any sampling of a picture element period. An output 11 of the mem- 
method other than the method shown in the drawing ory P is connected to a first input 13 of an adder 15, of 
can also be used; for example, it is not necessary to which an output 17 is connected to a second input 19 of 
sample from line to line in anti-phase. 25 the adder 15 via a memory L with a delay of a line 

The picture signal may be temporally low-pass fil- period. An output 21 of the memory L is connected to 
tered previous to the motion estimation. This has the an output 23 of the spatial filter, 
advantage that high vertical frequencies in a picture, An embodiment of a motion estimator according to 
due to which large differences between subsequent the invention is shown in FIG. 6. An input 61 of the 
fields of the picture occur that are not caused by mo- 30 motion estimator is connected to a first two-phase sarn- 
tion, are being filtered out so that a better estimation pling circuit 63, and to a second two-phase sampling 
result is obtained. The mode of operation of a preferred circuit 69 via a field memory F. Each of these two- 
embodiment of the temporal prefilter is shown schemat- phase sampling circuits 63 and 65 is connected to a 
ically in FIG. 4. FIG. 4 consists of two halves, sepa- first-phase candidate motion vector determining device 
rated by a dotted line, each showing schematically a 35 67 and a second-phase candidate motion vector deter- 
number of lines from three subsequent fields of which mining device 69. The first-phase detennining device 67 
the intermediate field is the present field to be filtered. estimates the motion for the samples indicated by a *T* 
The filter operates with coefficients (i, £, J), wherein, in FIG. 3, and the second-phase determining device 69 
from field to field, lines from both neighbouring fields, estimates the motion for the samples indicated by a "2" 
lying alternately above or below a line of a present field, 40 in FIG. 3. The candidate motion vector determining 
are being multiplied by the coefficient J. In practice this devices 67 and 69 are connected to a best vector selec- 
fijter appears to lead to good results. tion circuit 71 and to themselves, via vector component 

The picture signal may be spatially low-pass filtered, memories 73, 75, respectively, for the storage of new 

previous to the motion estimation, to reduce influences starting vector components. As is described in the fore- 

of noise. The picture signal may also be spatially sub- 45 going, for the first-phase motion estimation a different 

sampled, which allows for a field memory to have only starting vector is used than for the second phase motion 

half the capacity needed otherwise. estimation. The best vector selection circuit applies to 

If components of the selected vectors are spatiaDy an output 77 of the motion estimator the best vector 
low-pass filtered, a vector field without abrupt tran- from the candidate motion vectors estimated by candi- 
sients is obtained which, inspite of the inaccuracy SO date motion vector determining devices 67 and 69. As 
caused thereby, appeared to provide a finer result, be- can be seen from FIG. 6, inputs of the vector compo- 
cause errors in the form of blocks occur less often in a nent memories 73 and 75 are connected to outputs of the 
picture having been interpolated using the obtained respective candidate motion vector detennining devices 
vectors. Through the averageing effect of the low-pass 67, 69. This configuration according to the invention, in 
filtering, even a sub-pixel accuracy may be obtained, i.e. 55 which, starting from each starting vector, a separate 
an accuracy which is finer than the distance between candidate motion vector is determined and a motion 
two subsequent picture elements. The low-pass filtra- vector is selected from both candidate motion vectors, 
tion of the estimated motion vector components fur- and in which, as components of the starting vectors, the 
thermore provides the advantage that low-pass filtered corresponding components, respectively, of the candi- 
vector components can be spatially sub-sampled, with- 60 date motion vectors already determined and which 
out fold-back effects being produced. The sub-sampling respectively correspond to the starting vectors, are 
operation provides a saving in data, so that the method taken, appeared to furnish better results than a prior art 
according to the invention can make use of a small like configuration in which the inputs of the vector 
picture portion size, which has the advantage that ob- component memories 73 and 75 are connected to the 
ject boundaries may be followed appropriately, but has 65 output 77 of the best vector selection circuit 71, in 
the disadvantage that the number of resultant vectors is which consequently, as components of the starting vec- 
large, while the method may still be applied in an en- tor, respectively corresponding components of the se- 
ceding portion of a picture signal transmission or stor- lected best motion vector are taken. A candidate motion 
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vector which corresponds to a starting vector and is between a newly estimated vector and the filter output . 

already determined for, for example, a left-hand adja- (or alternatively, a. previously estimated vector) be- 

cent picture portion, should be understood to mean that comes too large, the newly estimated vector will proba- 

candidate motion vector that is based on a starting vec- bly belong to a different object in the scene moving in 

tor which itself is a candidate motion vector of a picture 5 another direction, or, stated in another way, the best 

portion immediately to the left of said left-hand adja- vector selection circuit 71 is switching between two 

cent picture portion. different convergence directions. In that case, averag- 

To implement the modification described hereinbe- big of motion vectors will only result in garbage, so that 
fore which used both an estimation based on previous the filtering action would better be disabled, 
field vectors and an estimation based on current field 10 it is to be noted that while in the above description 
vectors, each vector component memory (73 or 75) several modifications and extensions have been de- 
should comprise a first memory storing the candidate scribed in combination with the invention as claimed, 
motion vectors from the current field, a second memory ^j, c f these modifications and extensions might also be 
storing the candidate motion vectors from the previous m combination with other motion estimators. For 
field, and a switch for selectively coupling the output of 15 example, the described pre- and postfilters can be used 
the first memory or the output of the second memory to ^h olner mo ti on estimators as well. The applicants 
the candidate motion vector determining device (67 or hereby give notice that new claims may be formulated 
69, respectively). This switch is controlled by stored t0 such f eatures and/or combinations of features during 
estimation errors as described hereinbefore. tne prosecution of the present application or of any 

As has been explained in the foregoing, the picture 20 furthcr application derived therefrom, 

signal may be applied to the motion estimator via spatial ^ e claim: 

and/or temporal low-pass filters (see FIG. 7), If there is % A metnod of estimating motion of at least one 

both spatial and temporal filtering, the sequence spatial picture portion in an image of a picture signal, compris- 

filtering first, followed by temporal filtering has the . the of . determining a number of candidate 

advantage that the memory required for the temporal 25 motion vectQrs starting from a corresponding number 

filtering can be smaller. Furthermore, the output 77 of of startin vectorS( ^ selecting a motion vector out of 

the motion estimator can be connected to a spatial low- ^ determined candidate molion vect ors, characterized 

pass filter (see FIG. 8) and optionally to a sampling by fls ^p^, of the starting vectors , compo . 

c "^i"!' . _ f . ... . , n nents of candidate motion vectors already previously 

FIG. 7 shows part of another embodiment of a mo- 30 detemined which motion ^ ^ 

Uon estimator according to sp 0n d respectively to the starting vectors, and which 

median prefilter is placed ^« « candidate motion vector components correspond re- 

vector determining circuit. This median prefilter can be . , ^ . ^ . ^ r ■ A 

' ' ° . ^ ^ A r.uZ- ~fVTr a t« spectively to the starting vector components, 

considered as an alternative to the prefilter of FIG. 4. In f ^-JLi :« i «,v*«^ 

FIG.7th^ 35 ^^^^ 

inputs of a median filter 79. Median filters as such are sub-sampling the picture portion m a first and a second 
known in the art, see U.S. Pat. No. 4,740,842 (PHN P*™- afirst candidate motion vector being determined 
1 1,613). An output of the median filter 79 and the input 40 f ° r * ne g fOU ? and r a s <*° nd candidate motion vec- 
61 are coupled to respective inputs of a.candidate mo- tor being determined for the second group, 
tion vector determining circuit 81, which may comprise 3- A method as claimed in claim 1 wherein a firs 
the combination of the candidate motion vector deter. component of a first starting vector is token to be equal 
mining device 69 and the vector component memory 75 to a first component of a first candidate motion vector 
as shown in FIG. 6. Due to this prefilter, a kind of 45 already previously determined for a first picture por- 
quasi-progressi ve scan signal is applied to the candidate tion, a second component of said first starting vector is 
motion vector determining circuit, so that estimation taken to be equal to a second component of a second 
errors caused by interlace are mitigated. candidate motion vector already previously determined 

FIG. 8 shows a spatial postfilter for the estimated for a second picture portion, while the first and second 
motion vectors. The output 77 of the motion estimator 50 picture portions do not coincide, and m that a first com- 
(see FIG. 6) is coupled to a weighted averaging circuit ponent of a second starting vector is taken to be equal to 
comprising a first multiplier 83, an adder 85, a delay 87, a first component of a third candidate motion vector 
and a second multiplier 89 coupling the output of the already determined for a third picture portion, a second 
delay 87 to a second input of the adder 85. The output component of said second starting vector is taken to be 
of the postfilter is formed by the output of the delay 87, 55 equal to a second component of a fourth candidate 
but may alternatively be formed by the output of the motion vector already determined for a fourth picture 
adder 85. Multiplier 83 multiplies by a factor 1-k, and portion, while the third and fourth picture portions do 
multiplier 89 multiplies by a factor k, According to a not coincide. 

preferred embodiment, both multipliers are controlled 4. A method as claimed in claim 3, wherein the third 
by a non-linear device including a subtracter 91 and a 60 and the fourth picture portions coincide with the sec- 
threshold circuit 93. The subtracter compares a newly ond and first picture portions, respectively, 
estimated motion vector to the filter output, and if the 5. A method as claimed in claim 3 wherein the first 
difference between these two vectors is larger than a and the second picture portions have a smaller y-coor- 
given amount, threshold circuit 93 sets the multiplying dinate than the picture portion for which the motion is 
factor k, which normally will be J, to zero thus effec- 65 being estimated, the first picture portion having a 
tively disabling the filtering operation so that the newly smaller x-coordinate and the second picture portion a 
estimated vector is outputted unfiitered. This switching larger x-coordinate than the picture portion for which 
action is based on the recognition that if the difference the motion is being estimated. 
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6. A method as claimed in claim 2, wherein the pic- . 14. A method as claimed in claim 4, wherein the 
lure portions in a picture for each row of picture por- picture portions in a picture for each row of picture 
tions are picture element blocks which are shiftedly portions are picture element blocks which are shiftedly 
stacked stacked. 

7. A method as claimed in claim 1, wherein the candi- 5 15. A method as claimed in claim 5, wherein the 
date motion vectors are determined in accordance with P«*« P^ons in *P^te^™**^ 
a criterion that provides that only a candidate motion P~ P icture dement blocks whlch ™ shlftedly 
vector which differs from the starting vector can be s £ A ^ m claim 15 wherein ^ 

determined 10 candidate motion vectors are determined in accordance 

8. A method as claimed in claim 1, wherein compo- ^ fl criterion ^ prov i des that only a candidate 
nents of the selected vectors are spatially low-pass fil- mQtion vector which dijfrers from the starting vec tor 
tered. ^ be determined. 

9. A method as claimed in claim 8, wherein said low- J7 A met iiod as claimed in claim 16, wherein compo- 
pass filtering step includes the steps of 15 nents G f th e selected vectors are spatially low-pass fil- 

recursively filtering said selected vector components tered. 

to obtain output components; 18. A method as claimed in claim 10, wherein previ- 

comparing said output components to said selected ous to the motion estimation the picture signal is tempo- 
vector components to obtain a control signal indi- rally low-pass filtered utilizing a temporal filter having 
eating whether said selected vector components 20 the coefficients \, i, J, wherein from field to field, lines 
differ more than a given threshold from said output from both adjacent Fields, located alternately above or 
vector components; and below a line of a present field are multiplied by the 

disabling said recursively filtering step if said control coefficient J. 

signal indicates that said selected vector compo- 19. A method of estimating motion per picture por- 

nents differ more than said given threshold from 25 tion in an image of a picture signal wherein, starting 

said output vector components. from a starting vector, a motion vector is determined 

10. A method as claimed in claim 8, wherein the according to a previously defined criterion, character- 
low-pass filtered vector components are spatially sub- i*ed in that a first component of the starting vector is 

l e| j taken to be equal to a first component of a first already 

11. A method as claimed in claim 1, wherein, previ- 30 previously estimated motion vector, belonging to a first 
ous to the motion estimation the picture signal is tempo- picture portion, and a second component of the starting 

„ , ru , „«-r„: j r~ i f ar . v,„ - „ vector is taken to be equal to a second component of a 

rally low-pass filtered utilizing a temporal filter having * 1 * j J 

rc • ti i i\ v - a. r \a *r\ r.^A i;«J* second, already previously estimated motion vector, 

coefficients (i, J, J), wherein from field to field, lines • A L « ««♦ „„j 

- , ,' j u * 1 u belonging to a second picture portion, said first and 

from both adjacent fields, located alternately above or ° ^> „ ^iJliA^r* 

J . . , 4 . , , 35 second picture portions not coinciding, 

below a line of a present field are multiplied by the 2Q A method „ claimed in claim 19 , wherein -for 

coefficient 4 . -j each picture portion, in addition to said estimation, a 

12. A method as claimed in claim 1, wherein said second motion estimation is also performed, wherein for 
candidate motion vector determining step includes the ^ sccond motion estimation components of a second 
steps of: 40 starting vector are assumed to be equal to respective 

taking the median of said picture signal, a field- corresponding components of third and fourth, respec- 

delayed picture signal and a field- and line-delayed tively, already previously estimated motion vectors 

picture signal to obtain a filtered signal; and associated with third and fourth, non-coinciding picture 

estimating said candidate motion vectors based on portions, respectively, and starting from the second 
said picture signal and said filtered signal, 45 starting vector a second motion vector is determined in 

13. A method as claimed in claim 3, wherein the accordance with the said criterion, whereafter a best 
picture portions in a picture for each row of picture motion vector is selected from said first and said second 
portions are picture element blocks which are shiftedly determined motion vectors. 

stacked. * * * * * 
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